Papers with probabilistic models
Locally Typical Sampling (2023.tacl-1)
Copied to clipboard
| Challenge: | a discrepancy in probabilistic language generators has puzzled the language generation community for years . |
| Approach: | They propose a method for local typical sampling to enforce a conditional entropy criterion for probabilistic models. |
| Outcome: | The proposed method can provide new insights into why high-probability texts can be dull or repetitive. |
compare-mt: A Tool for Holistic Comparison of Language Generation Systems (N19-4)
Copied to clipboard
| Challenge: | Unlike machine translation, natural language outputs are nuanced and there are no clear yes/no distinctions about whether they are correct or not. |
| Approach: | They describe compare-mt, a tool for holistic analysis and comparison of the results of systems for language generation tasks such as machine translation. |
| Outcome: | The compare-mt tool is an open-source pure-python package that has already proven useful to generate analyses that have been used in our papers. |
Improving Knowledge Base Construction from Robust Infobox Extraction (N19-2)
Copied to clipboard
| Challenge: | Existing knowledge bases are incomplete, resulting in poor answers and incompleteness. |
| Approach: | They propose a method to extract Wikipedia infobox tables to populate an existing KB. |
| Outcome: | The proposed method improves accuracy and completeness of the final KB significantly compared to DBpedia's baseline method. |
Evidence-based Trustworthiness (P19-1)
Copied to clipboard
| Challenge: | Existing methods to estimate the trustworthiness of information sources are local in that they apply to a given claim. |
| Approach: | They propose a framework for estimating the trustworthiness of information sources in an environment where multiple sources provide claims and supporting evidence. |
| Outcome: | The proposed models show that they improve on baselines and show that the proposed models are more accurate than baselines. |
On Decoding Strategies for Neural Text Generators (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study suggests that decoding strategies may be more important than the model architecture itself when generating text from probabilistic models. |
| Approach: | They propose to measure changes in attributes of generated text as a function of decoding strategy and task using human and automatic evaluation. |
| Outcome: | The proposed study shows that decoding strategies do not always transfer across tasks . authors show that the differences in attributes are not always consistent across tasks, they say . |
A Japanese Word Segmentation Proposal (P19-2)
Copied to clipboard
| Challenge: | Current word segmentation methods may produce different segmentations for the same strings . this occurs when strings appear in different sentences . |
| Approach: | They propose to use Japanese word segmentation methods that use a morpheme-based approach to produce different segmentations for the same strings. |
| Outcome: | The proposed method produces much more consistent segmentation than the current morpheme-based one. |
On the Efficacy of Sampling Adapters (2023.acl-long)
Copied to clipboard
| Challenge: | Using sampling adapters can improve the quality of the generated text. |
| Approach: | They propose a framework for understanding sampling adapters and propose 'sampling adapters' they argue that the shift enforced by them can be viewed as a trade-off between precision and recall . |
| Outcome: | The proposed framework can be used to improve the quality of language models by modifying their distributions to improve their precision and recall. |
The problem with probabilistic DAG automata for semantic graphs (N19-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) annotations are directed acyclic graphs, but most probabilistic models view them as strings or trees. |
| Approach: | They show that some DAG automata cannot be made into useful probabilistic models by assigning weights to transitions. |
| Outcome: | The proposed model can't be made into useful probabilistic models by assigning weights to transitions . the proposed model is not feasible for all variants, but it is problematic for planar variants if they are not rooted . |
Accurate Legal Reasoning at Scale: Neuro-Symbolic Offloading and Structural Auditability for Robust Legal Adjudication (2026.acl-industry)
Copied to clipboard
| Challenge: | Legal texts contain computational legal clauses that exceed the semantic complexity of the realworld activities they govern. |
| Approach: | They propose a neuro-symbolic approach to legal adjudication using an LLM . they use a typed graph intermediate representation to translate a legal text into a deterministic contract language . |
| Outcome: | The proposed system reduces compute costs by over 90% in high-volume workflows while satisfying auditability requirements. |
Disambiguated skip-gram model (D18-1)
Copied to clipboard
| Challenge: | Disambiguated skip-gram is a neural-probabilistic model for learning multi-sense word embeddings. |
| Approach: | They propose a model that is end-to-end differentiable and can be interpreted as a feed-forward neural network. |
| Outcome: | The proposed model improves state-of-the-art in word sense induction benchmarks. |
Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for topic modeling learn topics with a flat structure . however, such methods have data scalability issues . |
| Approach: | They propose to use nonparametric neural variational inference to extract a tree-structured topic model with reasonable structure, low redundancy, and adaptable widths. |
| Outcome: | The proposed model extracts a tree-structured topic hierarchy with reasonable structure, low redundancy, and adaptable widths. |
Heuristically Informed Unsupervised Idiom Usage Recognition (D18-1)
Copied to clipboard
| Challenge: | Existing models for idiom usage recognition have failed to recognize usages without annotated examples. |
| Approach: | They propose an unsupervised method for recognizing the intended usages of idioms by using distributional semantics to identify literal usages. |
| Outcome: | The proposed method performs competitively against supervised methods. |
Benchmarking Neural Topic Models: An Empirical Study (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural topic modeling has been attracting much attention recently due to its ability to leverage the advantages of both neural networks and probabilistic topic models. |
| Approach: | They propose to evaluate neural topic models in three tasks using large datasets and a set of metrics to compare them. |
| Outcome: | The proposed models perform better in the first and third tasks than the traditional probabilistic models and are better in many cases. |
Lazy-k Decoding: Constrained Decoding for Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Specifically, we combine probabilistic models with constrained decoding approaches in structured prediction tasks. |
| Approach: | They propose a constrained decoding method called Lazy-k to combine probabilistic models with constrained methods in structured prediction. |
| Outcome: | The proposed method allows for more flexibility between decoding time and accuracy. |
Generalized Entropy Regularization or: There’s Nothing Special about Label Smoothing (2020.acl-main)
Copied to clipboard
| Challenge: | Prior work has explored regularizing the output distributions of probabilistic models to alleviate overfitting. |
| Approach: | They propose a family of entropy regularizers that have a connection to regularization . they find that label smoothing provably does not allow for sparsity in an output distribution . |
| Outcome: | The proposed method improves the relationship between model entropy and performance on language generation tasks. |
Effective Estimation of Deep Generative Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing techniques for parameterisation of probabilistic models by deep neural networks are difficult to use in language modelling due to posterior collapse. |
| Approach: | They propose to use variational auto-encoder to estimate probabilistic models of language by deep neural networks. |
| Outcome: | The proposed model performs reasonably well given enough resources, but a favourite can be named based on convenience. |
IntentionESC: An Intention-Centered Framework for Enhancing Emotional Support in Dialogue Systems (2025.findings-acl)
Copied to clipboard
| Challenge: | IntentionESC defines the possible intentions of supporters in emotional support conversations, identifies key emotional state aspects for inferring these intentions, and maps them to appropriate support strategies. |
| Approach: | They propose an Intention-centered Emotional Support Conversation framework which defines the possible intentions of supporters in emotional support conversations, identifies key emotional state aspects for inferring intentions, and maps them to appropriate support strategies. |
| Outcome: | The proposed framework defines the possible intentions of supporters in emotional support conversations, identifies key emotional state aspects for inferring these intentions, and maps them to appropriate support strategies. |
Prompting the Unknown: Understanding Response Uncertainty in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are widely used in decision-making across diverse domains. |
| Approach: | They propose a prompt-response concept model that explains the relationship between the amount of task-relevant information provided in the prompt and the LLM-generated response uncertainty by identifying four sources of response uncertainty. |
| Outcome: | The proposed model shows that the amount of information provided in the prompt influences the LLM-generated response uncertainty. |
CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling (2026.findings-acl)
Copied to clipboard
| Challenge: | Topic modeling seeks to uncover latent semantic structure in text corpora with minimal supervision. |
| Approach: | They propose a lifelong hierarchical topic model based on incremental probabilistic concept formation that constructs semantic hierarchies online without predefining the number of topics. |
| Outcome: | The proposed model achieves strong topic coherence, stable topics over time, and high-quality hierarchies without predefining the number of topics. |